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CAM RESEARCH PHILOSOPHY 

For some time it has appeared necessary to refocus the direction that edu- 
cational research has taken over the years. For many years educational research 
has empliasized tightly controlled experiments in which only one variable at a 
time is manipulated in an experimental group or groups, while control groups 
are not manipulated and remain as static as possible. 

Educators have tried to control many different variables in an attempt to 
Isolate the single variable that might have a causative effect on student per- 
formance. Most generally this approach haa been a dismal failure. One has only 
to examine the literature on any particular problem and note that in most Instances 
there is no clear cut indication that certain solutions are consistently better. 
For many problem solutions there are as many experiments producing no efiecfs 
and negative effects as there are experiments that indicate some positive effect. 
It's not difficult to understand the reason for this. There are too many vari- 
ables to control in ar. educational setting. Among these variables are the stu- 
dents, the instructional naterial, the teachers, time of day, length of class 
time, number of students, type of testing reinforcement patterns etc. In almost 
any experimental situation the researcher will try to control as many variables 
as possible, but the dynamics of the classroom situation just preclude success- 
ful experimental control really being achieved. 

Recently educational researchers have begun to explore the possibility of 
using multi-variable statistical analysis as a way to circumvent the problem 
of controlling variables. This technique shows promise but many of the proced- 
ures and applications still have to be developed before wide spread implement- 
ation of this approach will occur. 

While the previous considerations concerning educational research are cert- 
ainly important ones, perhaps the most important one that any educational 
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researcher should be* cognizant of the effect his work has upon the classroom 
teacher in terms of disrupting classroom routine. Educational research, and 
that includes all evaluation that is done to determine the effect on students 
of particular instructional technique, teaching material, or intervention, must 
be recognized itself as an intervention into the classroom. The most commonly 
recognized and quoted research effect that is guarded a.gainst, or to which abnor- 
mally good results are sometimes credited, is the Hawthorne effect. The very 
Introduction of the research effort with its attendant publicity and hoopla, often 
creates a short term benefit that is measureable and significant. There is an 
alteriiative negative effect that educational research and evaluation can impart 
to a classroom. It can disrupt the daily routine and operation of the classroom, 
upsetting both the students and the teacher. Educational researchers and evalu- 
ation people quite frequently are hit and run artists who totally disregard this 
aspect of their total responsibility • 

Today's society is demanding that much more emphasis be placed on evaluation. 
This emphasis gives educational researchers and evaluators more status in their 
attempt to document the learning process, and possibly more effective functioning 
of educational systems. What they often forget, however, as they focus on find- 
ing a solution to some educational problem, is that any treatment is only as good 
as the person who is using it. If research and evaluation are indeed to become 
a panacea for improved education then teachers must become experimentalist and 
empirical in their decision making. Teachers must become directly involved in 
research to make it c living, functional part of today's educational process 
rather than some educational researcher's ego trip collecting dust on university 
library shelves. 

Comprehensive Achievement Monitoring as it is conceived by the Sequoia Union 
High School District is a tool that enables teachers to function as researchers 
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and evaluatbrs. Itfl concepts include a curriculum defined by performance object- 
ives. These objectives serve as discrete items to be studied in curriculum pro* 
duct research or as the objects of educational process research. In fact their < 
measurement is a partial indication of the success of the educational establish- 
ment. These performance objectives are measured by test items that arc written 
or designed by teachers to measure student performance gn each performance ob- 
jective. Thus the CAM system, through Its test items, truly reflects what a stu- 
dent is supposed to know or be able to do as determined by the teacher, and not 
by some externally created standardized test. 

Sets of interchangeable test forms are created for each test. Sampling 
techniques are used to get estimates of what students know or are able to do 
without subjecting them to long involved testing situations that turn them off. 
No student has to respond to all the questions that may need to be asked in order 
to furnish the teacher, who is the researcher, with all the information he needs. 
Thus that very important factor of class disruption is kept to a minimum. Fur- 
thermore since the data is being collected relative to the regularly scheduled 
curriculum of the class, extra time for testing is cut to a minimum because there 
is no need to add extra tests to the students' schedules just to satisfy some 
externally located educational researcher. 

Two other concepts of CAM are periodic testing, usually weekly or biweekly, 
as determined by the teacher, and the storage of the collected information so that 
a longitudinal history of performance is the result. This approach builds a data 
bank that continually can be tapped for information concerning individual students 
and groups of students, all the time being unobtrusive as far as students are 
concerned, and part of the regular classroom routine. The massive task of data 
collation and data storage is done by computer. Without the computer the approach 
f:oward research and evaluation that has been described would be a physical impos- 
sibility, 
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If teachers are to function empirically they must have at their disposal 
quick feedback of information. It is very difficult either to modify curriculum 
or Instructional activities, or to work with students individually, without in- 
stant access to data. Computerized analysis and reporting of results on an over^ 
night basis to the specification of the teacher is another concept of CAM, 

Each of the above concepts is being developed to provide a complete evalua- 
tion cycle. The cycle is important in that we hope evaluation will become a 
continuous process that provides the teacher with a constant feedback of infor- 
mation that he can use to improve student learning. Of course there is a possible 
weak link in the cycle. The tool, CAM, can only be as t>ood as the teacher v/ho is 
using it. If teachers cannot successfully use the tool and interpret results 
then the evaluation cycle will never be completed and the tool Is useless as far 
as teachers are concerned. 

Naturally the tool is a new one. From our experience with CAM and our in- 
service training program we realize that there is a great amount of knowledge 
that teachers must have in order to implement CAM before they can use it effect- 
ively to make decisions about curriculum, instructional activites, and students. 

feel that teachers will be more prone to use data in their daily instruct- 
ional decision making if there is a particular problem that they are interested 
in exploring. Therefore we have asked each CAM lead teacher to think about one 
hypothesis that he or she would like to test in their classes this coming year* 
We are not requiring that this hypothesis have any concern with the total problem 
of the effectiveness of CAM, We think that if each teacher selects a hypothesis 
that he or she wants to study then these teachers will begin to study the data 
after each test administration in an attempt to find out whether their hypothesis 
is true or not. More importantly we hope they ultimately will positively affect 
the final result by something they might do as teachers in the classroom. At 
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that point teachers will be taking a very important step toward using CAM data 
as an everyday part of their teaching operation and the evaluation cycle will be 
closed. Evaluation will become functional and formative. 

What does this mean to educational researchers? Teachers need assistance. 
Most of them are well read only in their subject matter field. But they generally 
are creative. They need to be stimulated concerning the possibilities for what 
they can do within their classroom. Educational researchers can serve as tutors 
and guides to teachernr But they must alsways keep in mind that unl(?ss the 
teacher enthusiastically embraces what the educational researcher is saying, or 
wants to try, then the researcher wi] 1 be an albatross. The teacher is probably 
the most Important variable in today's schools. \^nnat works for one teacher may 
not for another, and that is the most important thing for researchers to find 
out. Make the classroom and that teacher the global focus of your effort rather 
than the idea or theory you are intellectually excited about, and you'll event- 
ually build a mosaic that will give you answers to other questions. 
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II. WHAT IS CAM? 

A new design for evaluating classroom acliievemeri t levels has been 
operational in the Sequoia Union High School District since September 1970 . 
Called Comprehensive Achievement Monitoring (CAM) , the design *s methods 
were developed at Stanford University and the University of Massachusetts 
under a grant from the Charles F. Kettering Foundation. 

The CAM design includes the following components: 

1. A curriculum defined in terms of performance objectives. 

2. Test items designed to measure student performance on each 
objective for the course. 

3. A set of comparable test forms which evaluate performance 
on all or a sample of all the objectives in the curriculum, 

4. Testing throughout the period of the course at intervals 
determined by the teacher. 

5. Computerized analysis and reporting of results within a few 
days after each test administration. 

6. Interpretation of results by teachers and students leading 

to decisions on curriculum, instruction, and study priorities, 

7. Modification of curriculum, instructional activities, and 
components of the CAM design. 

Each of these components is an integral part of the evaluation cycle. 
Evaluation becomes a con*:inuous process which provides the teacher 
with constant feedback of information which he can use to improve the 
learning process of the students. 



* Sequoia is pioneering the use of CAM, being the only district in 
California and one of three in the Western United States using CAM 
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Strengths of CAM 

Pre-lnstructlon measures .' In a CAM design objectives can be 
pretested prior to instruction. If the students have acquired information 
and skills related to some objectives from outside sources the teacher 
will discover this from the pretest data and will not have to cover 
the miiterial in the course* Coritinuous testing of objectives to be 
taught later in the course may indicate their relationship to objectives 
currently being taught. Changes in performance of specific objectives 
may indicate that the teacher should make alterations in the course; 
the sequence of instruction could be changed, the amount of time 
spent on certain objectives could be changed, and/or some ohjectives 
could be eliminated from the course. 

Immediate post-instruction measures . The usual classroom test uses 
only one test item lor each objective to evaluate material which has just 
been taught. To estimate achievement levels on just-taught objectives 
CAM may use more than one item for each objective. In addition, since 
sampling techniques are used, each objective can be tested without 
increasing the length of the test* Thus, with item sampling, objective 
sampling, and student sampling, a CAM test produces more information 
than the usual classroom test. 

Retention measures . Particularly important for the teacher is the 
ability to make decisions concerning that part of the curriculum that 
has already been taught. Because of this, there is a continual testing 
for retention of material which was previously learned. Intervals 
between *^teach" and "test*' times are recorded so that analysis of 
retention can be made for known intervals of tirae. The teacher decides 
how many days elapse before the data represent retention. Retention 
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data can be used to alter the instructional process in a meaningfull manner. 

The teacher makes decisions . A fundamental principle in our 
district's use of CAM techniques to evaluate the effectiveness of 
instruction is that decisions are made by the teacher. The items 
which are used to measure the achievement of these objectives are 
also determined by the teacher. Items may be either of the paper 
and pencil type or those that require observations by a reliable rater. 
Examples of types of items which may be used are: 

1 . True-false 

2 . Multiple choice 

3. Completion 

4. Computation 

5. Essay 

6. Problem solving 

7. Psychomotor observations 

Other decisions which teachers make in our CAM program include when and 
how often test monitors will be given. 

Information f eedback * Although the design of the testing 
procedures in the CAM system can become as complex as a teacher desires, 
the computerized information supplied to school personnel and students 
is easily read and understood. 

For individual students it includes — 

a. The total score on the current test and on each 
previous test. 

b. The score on only those items for which instruction 
has occurred on the current test and on each 
previous test . 
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c. The student's response to each item on the 
current test reported by objectives rated as 
correct (+) , Incorrect (-) , or no responBC 
(0). 

d. Whether or not the student had been Instructed 
on the item. 

For any group of students in the course it includes — 

1. After each test administration: 

a. The total percentage correct for each student 
for each test administration, 

b. The percentage correct on instructed objectives 
for each student for each test administration. 

c. The percentage correct for all students on 
each objective and any specified group of 
objectives , e.g.;, unit , for each test admin- 
istration. 

2, Upon demand, e.g., end of quarter or semester: 

a. An item analysis for each item as it measures 
achievement before instruction, immediately 
following instruction, and on a retention basis. 

Courses in CAM . During the first semester of the 1972-73 
school year seventy courses are being monitored by CAM. Many 
content areas are represented with courses in mathematics, biology, 
chemistry, physics, earth science, geography, economics, government, 
social psychology, anthropology , social studies, history, safety 
education, child development, reading, vocabulary, english, music, 
art., physical education, metal work, drafting, foods, Spanish, 
French, business law, marketing, accounting, and typing. Seventy- 
five teachers and approximately 6,500 students are participating in 
the program. 
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III CAM COMPUTERIZED FEEDBACK 

There are five basic types of computerized feedback: individual 
student reports, group summary reports, teacher summary reports, 
form analyses, and curriculum analyses. 

Figure 1 is an example of an individual student report. Each 
student receives a copy of his own student report after each test 
adminietratiou. The data shovm in Figure 1 is for Tim Boetticher's 
second test adminis\:ration. The left portion of the report tells 
the student for each item on the test what objective the item was 
associated with, whether each item on the test W£'S right (+) or 
wrong (-) , and if instruction was completed on that objective (YES). 
The right portion of the student report summarizes the student 
performance on the present test administration and all previous 
test administrations. He is given two scores, the percentage 
correct on the total test and the percentage correct on those 
objectives with instruction completed. 

A teacher summary report is shown in Figure 2, This report 
provides the teacher with each student ^s performance on each test 
administration, the test form the student was administered on each 
test administration, the cumulative average for each student on all 
test items thus far (TOT) and the cumulative average for all items 
for which instruction was completed prior to the test administration 
(YES). 

The Group Summary Report is used to present percentage correct 
for any specified set of objectives, e.g., unit or all objectives. 
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for any specified group of students. Each teacher usually gets one 
group summary for all students in the course, one for all his students, 
and one for each class of his students. The Group Summary Report shown 
in Figure 3 is for Mr. Castoro's c?.ass that meets during Period 7. 

The Group Summary Report also shows the performance of the 
student& on each objective. From this report Mr. Castoro will realize 
that he needs to do some extra work with the students on Objective 107 
since the student performance (15%) was fairly low relative to the 
other objectives. 

An example of a form analysis is shown in Figure h» After each 
teet administration, the teacher receives a form analysis for each 
form used during that test administration. The response data from 
all students in the course who received that form are included in the 
analysis. The print-out shows the objective number, the correct 
response alternative, the average percentage correct, and the per- 
centage of students who chose each response for each item on the test 
form. 
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Practical Application of CAM n« Practiced at Carlmont High School 

in the Algebra I Program 

Russel] Reed 

Now in Its third year, the application of CAM in the Algebra I program at 
Carlmont has developed from an experimental basis to an integral part of the 
course. Modifications of course design and changes in the evaluation procedures 
have evolved to a point where those involved clearly understand what is expected 
of both teacher and student. This year four teachers and approximately 250 
studentH participate in the program. 

Course Design 

Sixteen units (chapters) written by Albertus Niehuis Jr., head of the 
Carlmont Math Department, coveriiig a years span of topics from sets to the 
development of the quadratic formula make up the course offering. The topics 
included in the first ten units and the number of objectives and items written 
for each unit are shown in Table 1. On the surface the course is group paced. 
However, because of the use of performance objectives and the feedback inherent 
in the CAM process, the course could be considered as a teacher centered 
individualized instructional program. Each unit to be taught is preceded by a 
set of performance objectives which identify the basic goals of the course. 
Examples of three objectives a\nd their associated test items are shown in Table 
2. The objectives are also stated throughout the unit, making sub-topics easily 
accessable to the student for study purposes. 

Evaluation Design 

The evaluation design used for Algebra I is referred to as a Sliding Unit 
CAM, so called because each test administration has components related to 
retention, immediate post-instruction, and pre-instruction test items. A test 
administration given after instruction on Unii 5 would have test items pertaining 
♦ 
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to Units A, 5, and 6; a test administration after instruction on Unit 6 would 
test materials from Units 5, 6, and 7, etc. Each test administration, given at 
wwo to three week intervals, consa^ists of three test forms. Two of these forms 
are used on a predetermined testing date and differ only in terras of retention 
and pre-instruction test items. The third fonn with parallel test items is 
used exclusively for retake purposes. Students are requested to take the third 
form if they do not meet the minimum standards of performance as determined by 
the teachers. The vobjectives Included on each test form for the test administra- 
tion related to Unit 5 are shown in Table 3. The students are also given a 
final exam which contains a sampling of objectives from the entire semester; the 
students are told which objectives will be included on the semester final. 

Results 

Each student receives a copy of their individual print-out. Examples of 
these reports are shown in Figures 1 and 2, When necessary, the teacher and 
the student can examine the print-out and the student can get individual 
consultation on the objectives that need more study. 

The teacher receives a group summary report for each test administration. 
An example of the group summary report is shown in Figure 3. These reports lead 
to the possibility of general comparisons among teachers and also give information 
that clearly indicates the weaknesses of the class as a whole on objectives 
instructed and strengths on objectives not yet instructed. 
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Sample Objectives and Test Items 

Objective 0405: The student will select the correct simplification 
for a complex function. 

Tost Item 040502: 

Select the correct simplification for 2 + ^ 

3 

6 6 



Objective 0502: The student will select the correct geometric 
description from a worded description of a set 
of numbers- 

Test Item 050202: 

Select the correct geometric description for all 
numbers greater than 5 or less than or equal to 1. 



1. 



3. ^ (>a — y 5. ^ co^ 



2. ^ y 4,o— • 

Objective* 0605: Given an expression, and the replacements for the 
variables, the student will select the correct 
evaluation. 

Test Item 060502: 

2 

For the expression 2in - n , evaluate correctly when m = -5 
and n = 2 . 

1, -14 3. 6 5. 4 

2. -6 4. 14 
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Form 51 



Tabic J 

KvaluaUion Design - Algebra 1 
Form 52 



Form 53 



QN 


Objec tivc 


QN 


Objective QN 


Objec tiv 
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0401 


1-2 


0501 
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050102 
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g 0402 
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3-4 


0502 
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050103 
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^ 0403 
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(S 0404 
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0505 
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050303 
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13-14 
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M 0504 
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0402 
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u 0403 
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050504 


14-15 


(2 

0505 
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11 
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^ 0404 


12 


050604 


18-19 


0507 


19 


0405 


13 


050703 










14 


050704 


20 


0601 


20 


0601 






21 


0602 


21 


0602 






22 


0603 


22 


0603 






23 


2 0604 


23 


2 0604 
pd 






24 


0605 


24 


0605 






25 


0606 


25 


0606 






26 


0607 


26 


0607 
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SEABAUGH LANCE 663070 SECTN 12 REED 

CAlOl 

TEST ADM 4 - 10/18/72 
FRACTiON CORRECT ON ALL ITEMS IS 23/ 26 FORM 52 

FRACTION CORRECT ON UNIT ITEMS IS 14/ 14 



TEST FKN COR FRN COR 



QN 


OBJ 


RP INS 


QN 


OBJ 


riT 


INS 


ADM FORM 


TOTAL 


UNIT 


15 


401 


$+ YES 
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505 


$+ 


YES 


1 12 


23/30 
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402 
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26 
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s- 











CUM TOT 8S./113 53/ 56 



Figure 1. An individual student report for Lance Seabaugh. 
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STEWART TOM M 4A7 235 SECTN 12 REED 

CA?01 

TEST ADM 4 - 10/18/72 
FRACTION CORRECT ON ALL ITEMS IS 23/ 26 FCRM 51 

FRACTION CORRECT ON UNIT ITEMS IS 12/ 14 



TEST FRN COR FRN COR 
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CUM TOT 91/113 48/ 56 



Figure 2. An individual student report for Tom Stewart. 
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COMPREHENSIVE ACHIEVEMENT MONITORING - GROUP SUMMARY REPORT CAlOl 
CARU40NT ALGEBRA I 10/19/72 
STUDENT GROUP 7600 REED'S STUDENTS 



TEST ADMINISTRATION 
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Figure 3. A Group Summary Report for Mr. Reed's students. 
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COMPREHENSIVR ACHIEVEMENT MONITORING - GROUP SUMMARY REPORT CAlOl 
CARLMONT ALGEBRA I 10/19/72 
STUDENT GROUP 7600 REED'S STUDENTS 



TEST ADMINISTRATION 
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Figure 3 (cont'd) A Group Summary Report for Mr. Reed's students. 
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CAM Basic Reading Skills - Menlo-Atherton High School 

Mary Condon 

In a reading program such as Menlo-Atherton, has where there are 150 students 
involved, all reading two or more years below grade level, the first obligation 
of the teacher is to give these students tools. By tools I mean we need to teach 
word attack skills. If a student is not able to £ssociate sounds to letters, if 
he cannot blend letters, if he cannot see syllables and sound them out, if he 
cannot see the different parts of a word like preiixes and suffixes and know 
their functions, he is incapacitated and almost without exception a poor reader 
or even a non-reader. 

The teaching of skills has always been ?n important part of our reading 
program. However, in the past we used to present the material, practice it for 
a reasonable length of time, test on it, give grades, and then go on to the next 
phase of teaching. 

With the advent of CAM, the format of the program has been altered* We still 
teach the same material but it now has been categorized and applied to CAM. The 
basic word attack skills program includes 48 objectives of which 19 are oral 
objectives and 29 are written. An example of a written objective if? "Orally given 
a list of common two-syllable words ending in Le, the student will write the 
separate syllables." This objective and the associated words are presented in 
Table 1. An oral stimulus requires a written response; conversely, a written 
stimulus requires nn oral response. An oral objective sounds like this, "Given 
a list of common words ending w:^>th a silent the student will accurately 
pronounce the word^?." This objective and the associated words are shown in 
Table 2. Again, during the daily drill, the student learns that an e^ at the end 
of a word is silent and it also makes the preceding vowel sound long% During 



Table 1 

Example of a Written Objective 



19 Orally given a list of conimon two-syllable words ending 
In J^, the student will write the separate syllables. 



bottle 


cnndlo 


bumble 


gentle 


jungle 


fable 


ruffle 


marble 


sample 


title 


muzzle 


circle 


paddle 


maple 


cattle 


dimple 


tumble 


needle 


muscle 


pebble 



Table 2 

Example of an Oral Objective 

22 Given a list of closed syllable words, the student will 

add the final e and accurately pronounce the newly formed word. 



mad 


made 


fin 


fine 


mod 


mode 


sit 


site 


hop 


hope 


pal 


pale 


bit 


bite 


mat 


mate 


rid 


ride 


cut 


cute 


kit 


kite 


can 


cane 


fat 


fate 


hid 


hide 


hat 


hate 


slim 


slime 


fad 


fade 


slid 


slide 


pin 


pine 


rat 


rate 



each CAM test administration, each student takes an oral CAM and a written CAM, 
thereby receiving two computer print-out sheets, one for each CAM taken. 

When th€* results are returned to the student, they arc interpreted by the 
student and the teacher on a one to one basis. The student now has tangibJe 
results of his work. An example of an individual student report is shown in 
Figure 1. The student can actually see what he has done and what he further 
needs to do. These results also give the teacher specific information that she 
needs to work further with the student. 

The 48 objectives that make up our program cover a basic phonics or decoding 
course. Each day the students work in small homogeneous groups of six or seven 
for about 20 minutes drilling on the objectives that will be included in the next 
CAM. Multi-modal teaching techniques provide obvious advantages for educationally 
handicapped students. Use of visual, auditory, kinesthetic, and speech tactile 
learning modes enable the students to have successful learning experiences. The 
rema5.nder of the period is spend on vocabulary development and comprehension 
skills. The master schedule indicates that there is a test administration every 
two weeks. On each CAM there are some newly instructed items, some review items, 
and some uninstructed items. Last year after the third test administration, we 
noticed that the overall scores indicated that the students had not done as well 
as they had done on the first two test administration. After careful analysis 
of the results, it was concluded that the students had not mastered the objectives, 
probably because the material had become more difficult and there was now consider- 
ably more material to retain. Consequently, the decision was made to reteach 
the material and readminister the 300 series. This could be done without the 
students retaking the same test since each CAM has two test forms, and each 
student could take the alternate form which contained the same objectives but 
different items. Again because of the increasing difficulty of the material and 
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STEWART TOM M 45921A SECTN 1 CONDON 11/25/71 

MAAOl 

TEST ADM 5 - 11/24/71 
PERCENTAGE CORRECT ON ALL ITEMS IS 85 FORM 311 

PEFIIENTAGE CORRECT ON YES ITEMS IS 90 



TEST PCT COR PCT COR 
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the growing multitude of retention items* It was necesHnry to give two test 
adminlHtrations for the last two CAMS. The Immediate feedback of test results 
and their evaluation along with the prerogative of modifying the master schedule 
and the teaching strategy are in my opinion the reasons for the success of the 
program. 

Aft indicated in the final test results shown In Figure 2, a great deal of 
improvement can be made with such an instructional program. The forte of the 
program that tlie student Is never released from his obligation of mastering 
specific material because there is a constant overlapping of the material. CAM 
with its computer print-outs has activated an interest in achievement; It has 
created a motivation for accc/mplishment . The grade level distribution of 
students in September, January, April, and ^'une is shown in Figure 3. The 150 
students biegan the year with a vocabular^Y median grade level of 5.1; in May 
it had risen to 6.3. The comprehension median grade level was 4.7 in September 
and In May it rose to 6.9. These gains are remarkable indicating the overall 
effectiveness of the program. 
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Figure 3. Percent correct on CAM tests in September and 
February for students in the Menlo- Atherton 
High School Reading Program. 
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The Application of CAM for English - Notre Dame High School 

Marianne Rackham 



In order to understand the way In which we are using CAM in the Kngllsh 
Department at Notre Dame High School, It is necessary to first understand the 
situation in which we teach. Ours is a one track, college-prep high school 
which is operating on a flexible-modular schedule for the fifth year. Both 
these facts are factors in our CAM use. When we wer« originally approached by 
Mr. Easter, we had Just completed a year of work on a WASC evaluation. The 
WASC committee used Form B, which is the tool for curriculum evaluation, and 
so we were not unfamiliar with the writing of performance objectives, even in 
a field as esoteric as that of English. 

In the course of writing our objectives for the evaluation, I became more 
and more convinced that in a one track school like ours, we actually teach the 
same material over and over with an increasing degree of sophistication. We 
expect our ninth grader to learn — or relearn in many cases — grammar; we expect 
our sophomore to apply this knowledge to the writing of simple composition; and 
we expect our juniors and seniors to expand on the basic concepts even further. 
In every instance, we either presume a body of knowledge or we presume a lack 
of knowledge, but whatever the basic presumption, we teach and reteach the same 
students the same material year after ye^ar. To a certain extent, this same 
generalization can also be made about our teaching of literature. 

With this hypothesis as the basis of my CAM, I proceeded to design a course 
which would be operable on all levels and which would, hopefully, save us from 
teaching concepts which were in fact already a part of the students body of 
knowledge, and at the same time, prevent our assuming a body of knowledge which 
did not exist. I started by writing a number of objectives which began with a 
concept as basic as Identification of parts of speech and which progressed 
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through more basics like functions of parts of speech» capitalization, punctu- 
ation, recognition of sentence types, and culminated with a very sophisticated 
unit on language, connotation and denotation, paragraph construction, and 
finally the multi-paragraph composition. We initiated the program in the second 
senjester of acadeniic year 71-72 with our ninth graders, and the student response 
to the program was quite favorable. There were five teachers on the team — 
flexible - Inodular scheduling almost always involves team teaching — and 180 
students in the course. One teacher elected not to participate. Because of 
the late start and beause our projection for the year had not originally included 
the CAM material, we covered only n small percentage of the objectives in the 
original course. We did, however, cover enough material for me to recognize 
certain problems which existed in my original .evaluation design and over the 
summer, I re-wrote and re-vised. In September of this year, we initiated the 
course on three levels. At the outset, a Standard Cam — that is a test designed 
to sample all of the objectives in the course — was given to all the students. 
Using the material generated by the computer, every teacher was then free to 
determine the starting place for her class or classes. Teachers of junior and 
senior con^position have been amazed to discover that basics like puiictuatlon 
and sentence construction are areas which require work, while at the same time, 
teachers of certain freshmen sections have discovered these students are ready 
for composition assignments usually not presented in the ninth grade. 

When the student is presented with her printout, she is in a position to 
diagnose • hex own problems. She can tell, for example, if she has mastered a 
given objective by looking at the number correct out of the number attempted, 
and if she is not achieveing 80% or better, she comes for individualized 
instruction. Her teacher can tell from the data generated by the computer what 
kinds of exercises she should receive. In a class of mixed ability levels, it 
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l8 W'j practice to place a student who is achieving 80-100% on items we have 
taught in charge of a student who is achieving below the acceptable level. Both 
ore stimulated to achieve in this kind of n situation. 

This yesr, in addition to CAMming grammar and composition across three 
levels, we are also CAMming the literature portion of the Sophomore ' English course. 
This is a course which is team taught by four individuals, three of whom were 
involved in our original CAM projects. Our philosophy in this is very similar 
to our philosophy in our Grammar course. In the teaching of literature, we are 
basically always looking to teach certain basic objectives. It is only the 
degree of sophistication which differs. For example, characterization in the 
freshman short story unit is the same concept we teach in the senior novel 
course. Only our approach varies. Wi,th this in mind, the teachers involved 
wrote objectives and test items for each unit taught in English II. One teacher 
di:'signed a poetry course, one a novel course, one a drama course, and the fourth 
unit was given over to grammar. At Chis time, our objectives are largely 
concerned with basics, but it is our hope that with time and experience we can 
expand these units so that they too can be used across levels. The particular 
work of literature used would vary from class to class, but the framework into 
which it is placed would remain constant. All poetry courses are involved with 
meter and rime, language, allusion, diction, etc. All fiction courses are 
concerned with plot, character, setting, theme, and style. All drama courses 
are concerned with the content of poetry and fiction plus characteristics unique 
to the genre. Our goal for this year is to create a CAM framework into which 
we can fit most of the material taught in the English classes of our school. 

The data provided by the program enables us to assess our curriculum, our 
teaching methods, and our individual student performance. Once the assessment 
has been made, we can only improve. 
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CAM and Teacher Decision-Making 
Sequoia High School Geography Team 
Bruce Kdmondfl 



Introduction 

Comprehensive Achievement Monitoring has added a new dimension to the team 
Geography program at Sequoia High School. For the first time, we are able to 
raake important instructional decisions based on actual data rather than on 
intuition. As the Geography Team has become more expert in the use of this tool 
we have beocme more empirical in our approach to the problems of the classroom, 
more selective in our choices of activites and materials, and most important, 
more effective as teachers, as evidenced by marked increases in student progress 

Background 

Course Design 

The Geography Team is composed of four teachers and two teacher aides. 
There are six sections of team geography. Each period there is one large group 
class (55-70 students) which Is heterogeneous in composition and is directed by 
two teachers and two aides. There is also one advanced standing class (10-15 
students) each period with one teacher. Instruction is group-paced. 

The course content includes the following units of study: 

Unit 1 - Geography of Cities 

Unit 2 - Manufacturing and Agriculture 

Unit 3 - Cultural Geography 

Unit 4 - Political Geography 

Unit 5 - Habitat and Resources 

The text we use in the High School Geography Project: Geography in an 
Urban Age . 
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Evaluation Design 

The Sequoia Geography Team uses a ''sliding CAM" upon which is superimposed 
a semester ''standard CAM," That is, for each of the twelve test adinlnistrntions, 
students receive some pre-test items, some post-test items, and some retention 
items. Every objective (and there are nearly 200 for the year) is tested three 
times with three different test items. Standard CAM's are given as comprehensive 
semester examinations. The evaluation design for Geography is shown in Table 1, 

Teacher Decision-Making 

Immediate Decision-Making 

CAM allows us to make daily strategy decisions on the basis of hard data 
evaluation, Pre-^instruction testing has provided data for short-term decisions 
regarding instruction and classroom management, 

1, Instructional emphasis placed on objectives is determined 
by an evaluation of CAM pre-test summary data, 

2, "Exceptional" students are identified so that we may 
individualize our instruction. 

Post-instruction data allow us to: 

1, provide remedial prescription on an individual- basis, 

2, make decisions on material for which review is necessary, 

3, grade students on the object:ive basis of achievement. 

Long-Term Decision-Making 

While the use of CAM data for daily instructional decisions is at the 
heart of the CAM project, we are learning how to use the tool in a broader sense 
as well. The Sequoia Geography Team has sough answers to a number of questions. 
In each case, we defined a research hypothesis and CAM provided the vehicle for 
its measurement. Consider the following examples: 

Question 1: Are the High School Geography Project materials more effective than 
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Table 1 

Sequoia Geography - CAM Evaluation Design 
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the traditional geography textbook? 

CAM developed and tested a model which compared the effectiveness of HSGP 
materials to the traditional text using student performance as the measurement 
criteria. The following pattern appeared: 
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The HSGP materials are designed to be concept reinforcing, thus, the 
important difference in retention test scores. 

Question 2: Does the increased effectiveness of HSGP justify the extra 
cost of the materials? 

CAM designed a model which compared HSGP to traditional materials In terms 
of expenditures and student performance. Cost per unit of student performance 
gain favored the new materials. In addition, since the study was broken down 
by each seperate objective, we were able to determine those objectives which 
were prohibitively expensive in terns of student performance gain. 
Question 3; Of the many different activites and teaching techniques which we 
employ in the classroom, which ones seem to be most eff^ective? 

By documenting the teaching method used during the instruction of each 
objective we were able to compare the method against student progress data for 
each objective. 

Product Development - A Case in Point 

At the end of each year CAM participants are encouraged to carefully analyze 
their courses, rewrite and reorganized objectives, and to revise and reconstruct 
tests and evaluation designs. To do so, CAM produces test form analysis material 




and a variety of other summary data by which decisions on product development 
take place. 

As we reviewed our CAM data last spring, a problem became apparent. The 
data revealed that the student performance distribution curve was no longer bell- 
shaped, but bimodal. 



Number of 
Students 




0 100 
Percent Correct on Posttests 

Figure 1, Distribution of students on the posttests. 

Increases in minority students due to the diistrict's desegregation effort, 
and a steady growth of our Mexican-American population contributed to the curve 
distortion. 

Since our instruction is group paced, we had focused our instructional 
level at a middle point, providing individualized help for students at either 
end. The error in this approach is at once apparent. There were very few students 
at the instructional level. The questions, then, were 1) toward which mode do 
we adjust the level, and 2) how do we provide for the remainder? For many reasons, 
we decided to move the level downward thus providing for our ever-increasing 
number of educationally handicapped students and to establish advanced standing 
geography classes. 

Additionally, with the realization that special attention must be paid to 
the large group of low achievers, the team added a black teacher, an additional 
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teacher aide, and an earphone reading lab. We also rewrote much of the reading 
material to a lower vocabulary/comprehension level. 

The effects of these changes are being studied this year as we raise such 
questions as: 

!• What are the effects of withdrawing the advanced standing students 
from the team class a) on the large group and b) on the advanced 
standing group itself? 

2. Is there both progress improvement and attitudinal improvrment , 
or neither? 

3. Will the instructional intensification at the lower level result 
in significant student progress improvement? 

Without the data that Comprehensive Achievement Monitoring is about to 
provide, the kinds of instructional modification that we have made in the 
Sequoia geography program would not have been possible. The need for the changes 
would not have been as apparent, the specific problems to which we addressed 
ourselves could not have been isolated, and the results of the changes could 
not have been as effectively measured. 
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